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About Our Project 


• Brand-new IP 

• Alternate history 
19 th -century London 

• PS4 Exclusive 

• ~100 developers 
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About Our Project 


• Work began in early 2011 

• Originally just 2 graphics 
programmers 

— Now we have 5! 
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Engine Overview 

• Latest version of our in-house engine 

• Windows(DXll) and PS4, but heavily geared 
towards PS4 

— Fine-grained task scheduling 

— Low-overhead, multithreaded cmd buffer generation 

— PS4 Secret Sauce™ 

• Physically based rendering from the start 
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Engine Overview 

• Core tools framework uses C++ and Qt 

— Built on common Ul core 

— Can be embedded in Maya 

• Custom build pipeline built in C++ 

— Major engineering effort 

— Heavily multithreaded 

— Processed assets cached on local servers 
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Engine Overview 


• Renderer embedded in Maya 

— Uses DX11 Viewport 2.0 override (Maya 2014) 

— Render using their device instead of our own 

— Tell Maya which Ul to draw 

• Materials/particles/lens flares/etc. can be live 
edited inside of Maya 

• Serves as primary level editor 
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Memory 


•2 GB texture budget 

• 128 MiB sound budget 

• 700 MiB level geometry 

• 600 MiB character textures 

• 250 MiB global textures (FX, Ul, light maps, etc..) 

— Even characters have lightmap data 

• 700 MiB animation 
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Memory 

• Environmental art is mapped at 512 pixels per unit 

• Standard environment tilling texture is 1024x1024 
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Lighting Pipeline 

• Tiled forward lighting (AKA Forward+) 

— Depth prepass 

— CS bins lights per tile 

— Two depth partitions 

— Punctual light sources 
(no area lights. ..yet!) 
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Lighting Pipeline 

• Transparents fully lit with diffuse + specular 

— Separate depth prepass + depth buffer for transparents 

— Transparent light list generated per-tile 

• TileMinDepth = TileMin(transparentDepth) 

• TileMaxDepth = TileMax(opaqueDepth) 

— Also support per-vertex lighting for particles 
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Lighting Pipeline 


• Lightmaps for static geo 

— Baked on GPU farm using Optix 
— H-basis for directional variation 

• SH probes for dynamics 

— 3 rd -order (9 coefficients) 

• Pre-convolved specular probes 

— Cubemaps rendered in-engine, 
convolved in a compute shader 
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Sun: Mid Day 


: Emissive Intensity: 50,000 HDR Exposure: -4.4 



Indirect Light Emissive Intensity 


R Exposure: -2.0 
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Lighting Pipeline 
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Indirect Light Emissive Intensity: 1000 HDR Exposun 


Day: Ambient 



Lighting Pipeline 
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Ambient Occlusion 


• Directional AO maps (H-basis) baked for characters 
and static geo 

• Static geo only uses it to occlude specular from 
probes 

• Dynamic geo applies AO to diffuse from SH probes 
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With Reflection Occlusion 


"V 
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Reflection Occlusion Visualization 



Ambient Occlusion 


• Dynamic AO from capsules skinned to characters 
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AO Capsules On 
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Core Shading Model 

• Default specular BRDF is Cook-Torrance 

— D term is GGX distribution from Walter et al. 

— Matching Smith G term derived in same paper 

— Schlick's approximation for Fresnel 

— Lambertian diffuse, balanced with specular intensity 
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Core Shading Model 

• GGX + Cook Torrance == Lots of math 

— But we have lots of ALU! 

• Can be optimized 

— Don't use trig 

— Fold Smith G term into denominator 

— See our course notes from SIGGRAPH 

• Have artists work with sqrt(roughness) 

— More intuitive, better for blending 
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// Helper for computing the GGX visibility term 
float GGX_V1 (in float m2, in float nDotX) { 

return l.Of / (nDotX + sqrt (m2 + (1 - m2) * nDotX * nDotX)); 

} 

// Computes the specular term using a GGX microfacet distribution, m is roughness, n is the surface normal, h is the 
// half vector, and 1 is the direction to the light source 

float GGX_Specular (in float m, in float3 n, in float3 h, in float3 v, in float3 1) { 

float nDotH = saturate (dot (n , h) ) ; 
float nDotL = saturate (dot (n , 1)); 
float nDotV = saturate (dot (n, v) ) ; 

float nDotH2 = nDotH * nDotH; 
float m2 = m * m; 

// Calculate the distribution term 

float d = m2 / (Pi * pow (nDotH * nDotH * (m2 - 1) + 1 , 2 . Of ) ) ; 

// Calculate the matching visibility term 
float vli = GGX_V1 (m2 , nDotL); 
float vlo = GGX_V1 (m2 , nDotV) ; 
float vis = vli * vlo; 

// Multiply this result with the Fresnel term 
return d * vis; 

} 


n 
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Core Shading Model 


• Other BRDFs available 

— Beckmann (also taken from Walter et al.) 

— Anisotropic GGX 

— Hair (Kajiya-Kay) 

— Skin (pre-integrated diffuse) 

— Cloth 
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Skin shading 




Skin 


• Our most expensive shader! 

— Texture lookup per light based on N dot L 

— Multiple specular lobes 

• Didn't use shader gradient-based curvature 

— Too many artifacts 

— Used curvature maps instead 
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Skin 


Pre-integrated skin diffuse with SH 
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Ambient Skin (Diffuse only) 

Left is using normal SH lighting convolved with cosine kernel 
Right is using SH lighting convolved with the scattering kernel 
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Hair Shading 


• Not physically-based © 

• Kajiya-Kay 

— Haven't had time to investigate 
real-time Marschner 

• Tweaked Fresnel curve 

• SH diffuse using tangent 
direction 

— Analytic Tangent Irradiance 

Environment Maps for Anisotropic Surfaces[Mehta 2012] 
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Hair Shading 


Secondary specular lobe 

— Shifted along tangent towards tips 

— Takes albedo color 





Hair Shading 


• Shift maps to break up highlights 
— Additional shift along tangent direction 
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Hair Shading 


• Flow maps to define tangent direction 
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Cloth Shading 




Cloth Shading 

Observations from photo reference: 

• Soft specular lobe with large smooth 
fa 1 1 offs 

• Fuzz on the rim from asperity 
scattering 

• Low specular contribution at front 
facing angles 

• Some fabrics have two toned 
specular colors 



RFadyntbawn 

3 STUDIOS 



Cloth Shading 


• Inverted Gaussian for asperity 
scattering 

• Translation from origin to give 
more specular at front facing 
angles 

• No geometry term 
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Specular Aliasing 
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Specular Aliasing 

• Modify roughness maps to reduce aliasing 

• Using technique based on "Frequency Domain 
Normal Map Filtering" by Han et al. 



R£>adyRtbawn 

3 STUDIOS 




Specular Aliasing 

• Represent NDF as spherical Gaussian (vMF distribution) 

• Approximate BRDF in SH as a Gaussian 

• Convolution of two Gaussians is a new Gaussian 

• Use relationship to compute new roughness 
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Scanning 
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Material Pipeline 



RpadyRtbawn 

7 STUDIOS 



Material Pipeline 


• Materials = text assets 


• Uses our custom data language (radattr) 


• Language supports: 
— Types 
— Inheritance 
— Ul layouts 
— Metadata 


:bumpy_material := ( opaquematerial 
:enable_diffuse_ = true 
: enable_specular_ = true 

: specular_map = ( :name = M bumpy_material_spc” ) 
:normal_map = ( :name = M bumpy__material_nml M ) 

: specular_intensity = 0.05 
: specular_roughness = 0.1 

) 

:bumpy_material_red := ( bumpymaterial 
: albedo_tint_color = ( Color 
: r = 1.0 
:g = 0.0 
:b = 0.0 

) 

) 



Material Pipeline 

• Material authoring is feature-based 

— No shader trees 

• Material editor tool for real-time editing 

— Can also be hosted inside of Maya 
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Options 
Lighting 
Damage 
Blood Damage 


Skirt Blending 
Material FX 


Normal Map 
Tint Color 


Intensity 


Build Complete 


Material Pipeline 


Database Scene Viewer Options History Build Errors Editor 


ea test_metal 

(deferredmaterial) 

► BRDF 

► GI Baking Properties 


► Scorch Damage 


► Anisotropic Properties 

► Tessellation Properties 


layerO 

(deferred material) 
Surface Properties 


r Specular Lighting 


bumpy_material_nml 
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Material Pipeline 

• radattr inheritance used to make templates 

• Common parameters shared in base material 

• Derived material only stores changes from base 

• Quicker asset creation 

• Global changes can be made in a single asset 
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Material Templates 




Global Material Templates: Sample Metal Subset 




Global Material Templates: Sample Masonry Subset 




Common Materials: Production Samples 











Common Materials: Production Samples 
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Inherited 



Materials 




Inherited 



Materials 



Shaders 


• Hand-written ubershaders 
— Lots of #if's 

• Major material features = macro definitions 

• Parameters (spec intensity, roughness, etc.) hard- 
coded into the shader, except when animated or 
composited 

• Some code is auto-generated to handle textures 
and animated parameters 
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Shaders 


• Pre-defined "permutations" 

— Permutation = macro definition + entry point 

• Permutations for skinning, blend shapes, 
instancing, light maps, etc. 

• Debug permutation for visualization/debugging 

• Shaders compiled in build pipeline based on 
permutation + material 
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Shaders 


Compile Built 

Shader Set Shader Set 
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Shaders 


• Pros: 

— Shaders are optimized for a material 

• Optimizer has full access 

— No runtime compiling for the game itself (used by tools) 

• Cons: 

— Lots of shaders to compile! 

• We cache everything, but iteration can be slow 

— Monolithic shaders are hard to debug 

— Lean heavily on the compiler 
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Material Compositing 



Material Compositing 


• Mostly offline process 

• Material asset specifies composite "stack" 

• Each layer in the stack has: 

— Referenced material 

— Blend mask 

— Blending parameters 

• Recursively build + composite referenced materials 

• Combine each layer 1 by 1 using pixel shader 
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Material Compositing 

• Generates parameter maps from materials and 
blending maps 

• Support compositing subset of BRDFs 

— Cloth, GGX, and Anisotropic 

— Compositing cloth requires applying 2 BRDFs 
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Material Compositing 


Map 

R Channel 

G Channel 

B Channel 

A Channel 

Format 

1 

Normals X 

Normals Y 

N/A 

N/A 

BC5 

2 

Diffuse R 

Diffuse G 

Diffuse B 

Alpha 

(Optional) 

BC1 or BC3 

3 

Specular R 

Specular G 

Specular B 

Specular 

Intensity 

BC3 

4 

Roughness 

AO 

BRDF Blend 

Anisotropy 

BC3 
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Material Compositing 


// Compositing pixel shader run on a quad covering the entire output texture 
Compos iteOutput Compos itePS (in float2 UV : UV) { 

CompositeOutput output; 

float blendAmt = BlendScale * BlendMap. Sample (Sampler , UV) ; 
float4 diffuseA = DiffuseTintA * DiffuseMapA. Sample (Sampler , UV) ; 
float4 diffuseB = DiffuseTintB * DiffuseMapB . Sample (Sampler , UV) ; 
float diffuseBlendAmt = blendAmt * DiffuseContribution; 
output .Diffuse = Blend (diffuseA, diffuseB, diffuseBlendAmt, 

DiffuseBlendMode) ; 

// Do the same for specular, normals, AO, etc. 
return output; 

} 
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u prp_well_pump_c_tarnished_copper_source 

<mtt_coppw_poished_075) 


► BRDf 

► GI Baking Properties 

► Options 

► Lighting 

► Damage 

► Skirt Blending 

► Material FX 

► Anisotropic Properties 

► Tesseiation Properties 


„ layerO 

(mtt_copper_poh*hed_075) 
► Surface Properties 


+ 

Add Feabxe * 

m 


► Specular lighting 
’ Composite Material* 

► Material BASE, cmn_copper_a_tie_dark 

► Material BASf VARIATION WORN, cmn_copper_a_tIe_wom_ight 

► Material BASE VARIATION TARNISHED, cron_tamished_a_tae_Sght 

' ► Material TARNISHED, cmn_steel_a_tae_wom_dark 

► Material PRISTINE, cmn_copper_a_tie_pristine_Bght 

s » Material DIRT, cmn_dirt_fine_a_tBe_tan_dark 

r ► Material - WET 50, cmn_water_dan»pness_050 
- ► Material WET, cmn water dampness 100 


IIII1111131I 




I prp_well_pump_c_worn_enamel_black_sourc€ 

(cr»ri_eri«nel_fine_a_t<ejjrstr>e_black) 


► GI Baking Properties 

► Options 

► Lighting 

► Damage 

► Skirt Blending 


* Anisotropic Properties 
► Tesseiation Properties 


„ layerO 

(cmn_enamH_fine_a_tile_pri*tine_MadO 


► Surface Properties 


’ Composite Material* 

> Material BASE, cmn_enamd_fine_a_tfe_pristine_bla<Jk 

> Material BASE VARIATION WORN, onn_enamel_fine_a_tile_black 

’ » Material BASE VARIATION TARNISHED, cmn_enan»el_fine„a_tie_w. 

' ► Material TARNISHED, ann_steel_a_tae_wom_dark 

> Material PRISTINE, cmn steel a tle prtetine dark 
» Material RUST, cmn_nist coarse a_tie brown nux 

r ► Material WET, ann_water_dampness_075 

- » Material WET 50, cmn._water_dampness_050 
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• Up to 4 layers 

— Derived from base materials 

— Separate compositing chain 
per layer 

— Driven by vertex colors 


Material Layers 
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Material Layers 










Material Layers 


LayerParams combinedParams ; 


[unroll] 

f or (uint i = 0; i < NumLayers; ++i) { 

// Build all layer params from textures and hard-coded 
// material parameters 

LayerParams layerParams = GetLayerParams (i , MatParams , Textures); 


} 


// Blend with the previous layer using vertex data and blend masks 
combinedParams = BlendLayer (combinedParams , layerParams, vtxData , 

BlendMode , Textures); 


// Calculate all lighting using the blended params 
return ComputeLighting (combinedParams) ; 
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